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Several Race to the Top grantees are 
considering the development of video 
libraries as one element of their larger 
teacher and leader effectiveness reforms. 
Video libraries promise to offer “real 
world” examples of teaching practice 
that can be used for a wide range of 
purposes: providing teachers with 
concrete evidence of excellent practice 
aligned with college and career ready 
standards, serving as examples in inter- 
rater reliability training for observers 

of teaching practice, and so forth. This 
publication summarizes research about 
the specifications for video libraries in the 
Requests for Proposals (RFPs) developed 
by four States: Delaware, Illinois, New York 
and Rhode Island. 


This publication may be particularly useful 
for policymakers and administrators in 
State and local educational agencies that 
may be considering the development of 
a video library as a strategy for improving 
teacher and student performance. Below, 
the publication analyzes the contents 
and specifications of the RFPs that each 
State developed for video libraries. 

The differences between them offer 
insights into the different use cases for 
video libraries, as well as the ways States 
describe them. 


We ask the reader to keep in mind that 
this publication documents an emerging 
practice. It captures a moment in time 

in the design and procurement of video 
libraries by Race to the Top grantees, not 
necessarily best practice. 
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Background 


Early in 2012, the Delaware Department of Education 
(DDOE) requested research support from the Reform 
Support Network as it prepared an RFP for a video 
library that would be a part of an online assessment 
used to certify teacher observers. Delaware's teacher 
evaluation system—the Delaware Performance 
Appraisal System (DPAS Il)—is required in all Delaware 
public schools. In response to the DDOE's request, the 
Reform Support Network reviewed recent video library 
specifications for Delaware, Rhode Island, New York 
and Illinois and prepared a summary memorandum in 
March 2012, from which this publication is derived. 


The Rhode Island Department of Education (RIDE) 

is in the process of implementing a new teacher 
evaluation system, the Rhode Island Educator 
Evaluation Model, and sought to include video clips 
aligned to observable competencies in the evaluator 
training conducted in the summer of 2012. The 
intention is that, by illustrating each competency, the 
videos would help evaluators ground their ratings in 
real practice. 


The Illinois State Board of Education (ISBE) has 
selected the Consortium for Educational Change to 
develop a performance evaluation training program, 
deliver evaluator training courses, and provide 
technical assistance and web-based resources 
associated with the State's teacher and principal 
performance evaluation system. The video library is 
one feature of the web-based resources intended to 
support evaluator training. 


The New York State Education Department (NYSED) 

is seeking a partner to develop a video library to 
support its reform of teacher and leader practice. 

The goal is to establish a high-quality video resource 
library that contains footage of highly effective 
teachers and can serve as a professional development 
resource for educators across New York. 


Overview 


Each of the four Race to the Top grantees has a 
distinctive purpose for its video library. (The appendix 
provides a detailed analysis of each State’s RFP) New 
York's purpose is fundamentally different from that of 
the other three States. 


New York's video library is to be a resource for 
professional development, aligned to the New York 


State Common Core State Standards (CCSS), offering 


videos that demonstrate effective teaching (and 
leadership) aligned with the New York State Teaching 
Standards and New York State-approved rubrics, in 
various grades and subjects. New York's library will 
feature videos of principals as well 

as teachers. 


Illinois, Delaware and Rhode Island will primarily 
use their video libraries to train evaluators. All 
three States include “rater calibration” as a primary 
purpose for the repositories and will use their video 
libraries as an element of their evaluator certification 
process. Rhode Island and Illinois have also built 

in opportunities for “recalibration” to avert the 
phenomenon of rater drift, the tendency for raters to 
shift over time in their understanding of standards. In 
these States, raters will repeat the calibration process 
periodically in order to counter drift. 


A review of these RFPs offers useful perspectives for 
other States or districts contemplating video libraries, 
which could help them to ensure that the RFPs they 
write are designed to secure the product desired. 


Key Findings 


1. Each State defines the level of teacher 
performance the videos will document. New 
York’s videos document only the practice of highly 
effective educators. In contrast, the other three 
States’ video libraries will demonstrate the range 
of teacher performance: Delaware, across the full 
teacher performance spectrum; Rhode Island, 
across the State's four levels of performance; and 
Illinois, across a range of skill levels. 


. Each State requires the alignment of the 


video library with teaching standards. Illinois 
stipulates the alignment of the videos with the 
Illinois Professional Teaching Standards; Rhode 
Island, alignment with the Rhode Island Educator 
Evaluation Model's professional practice rubric; 
Delaware, alignment with the criteria from the 
State's teacher and specialist frameworks; and New 
York, alignment with the New York State Teaching 
Standards for teachers, the Interstate School Leaders 
Licensure Consortium Standards for principals, and 
the New York State-approved rubrics. 


. Each State specifies training in the use of the 


video library as a design consideration, although 
most training would not extend beyond the launch 
phase. (Delaware sought introductory baseline 

or assessment acclimation training; Rhode Island, 
orientation materials for teachers to convey the 
expectations and rigor of each competency; and 
New York, instructional coaches to support teachers 
with videotaping.) The Illinois Request for Secure 
Proposals (RFSP) included the most extensive 

focus on continuing training as part of the State's 
broader evaluation system to enhance teacher and 
evaluator development. 


. Only New York requires alignment of the 


videotaped lessons with student standards. 
The NYSED requires the video libraries to be 
aligned with the CCSS. The other States made no 
similar requirement. 


. Three States provide detail regarding subjects, 


grades and student populations to be featured 
in the videos. New York's RFP seeks videos that 
address all grades and subjects and showcase 
educators working in high-need schools, 
with students seeking a General Educational 
Development credential, students performin 
below grade level, over-age under-credited 
students, African-American and Latino adolescent 
males, girls in science and math classrooms, English 
learners and students with disabilities. The Illinois 
RFP also sought representation of a wide variety 
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of students and different preK-12 subjects and 
grades. Rhode Island is seeking a range of grades 
and content. Delaware's RFP did not name specific 
student groups or subjects or grades. 


. Three States align video library specifications to 
teacher and principal evaluation. Illinois plans an 
integrated approach to incorporating video libraries 
within the broader teacher and principal evaluation 
system. The State’s RFSP sought a partner to 

design and implement the evaluation system, of 
which the video library is one component. Illinois, 
Rhode Island and Delaware all propose to align the 
content of the videos and the supporting tools, such 
as observation rubrics, with the broader evaluation 
system. Additionally, they specified the need for 
videos that show diverse teacher performance 

levels aligned to the teacher effectiveness levels 
described in the State evaluation system. This 
intentional alignment of teaching quality with the 
evaluation framework is designed to support greater 
consistency across evaluators. 


7. Three States consider videos a tool for qualifying 


and/or certifying evaluators. Delaware and 
Rhode Island plan to use their video libraries as a 
tool to help certify evaluators to conduct teacher 
evaluations; Illinois and Rhode Island also seek 
alignment of the videos with the initial training/ 
qualification of evaluators. 


8. Three States address calibration of ratings/ 


standards. Illinois seeks pre-scored videos based 
on expert norms for rater calibration and regularly 
updated online resources to offer trainees frequent 
opportunities to calibrate rater reliability. Rhode 
Island requires that videos be aligned to observable 
competencies and seeks a tool to allow evaluators 
to recalibrate periodically. Delaware focuses on 
measuring the accuracy of evidence collection and 
decisions about performance levels. New York’s RFP 
does not explicitly address calibration. 
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9. Two States include details about the State role 


and selection criteria. Both New York and Illinois 
provide detailed information in their RFPs about the 
role the State will play as a partner in the proposed 
work and the criteria for evaluating proposals. This 
detailed information about roles and expectations 
serves to clarify from the outset the extent of State 
engagement and encourage proposals tightly 
aligned to the State’s desired outcomes. 


10. Two States provide detailed technological 


specifications. New York and Illinois both include 
more detailed technological specifications in 
their RFPs, such as information about the State 
server platform, preferred software for training, 
parameters for video quality, and the use of meta 
tags to identify lessons aligned with the CCSS. 


Appendix: Detailed State-by-State RFP Analysis 
P| delaware | Rhodetsland | NewYork lino 


Specifications/ . 
Guidelines 


Must be benchmarked to DE 
teacher frameworks 


Assessment training and delivery 
must be: 


Consistent in message/content 


Allow for flexible delivery (time/ 
location) 


Offered over period of time 


Accessible for refresher training/ 
assessment 


Must include “real world” video and/ 
or case studies across full teacher/ 
specialist performance spectrum 


Must address criteria in each of the 
four components of the teacher and 
specialist frameworks 


Must be of sufficient depth/breadth 
to ensure inter-rater reliability has 
been established 


Item pool for assessments must be 
large enough to allow for multiple 
test forms 
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Provide calibrated video tapes aligned 
to each rubric competency that 
differentiates across RI's 4 levels of 
performance 


Align observable competencies of the 
RIDE Model's professional practice 
rubric to a research observation rubric 


Provide a subset of calibrated 
videotapes and orientation materials 
for teachers to understand the 
expectations and rigor of each 
observable competency 


Develop a certification tool and process 
for evaluators to signal that they are 
initially trained and a tool that allows 
evaluators to recalibrate periodically 


Include system to track rater accuracy, 
reliability and potential drift 


Prefer training systems and 
assessments that can be integrated 
with RI’s newly developed Educator 
Performance Support System for 
multiple levels of users (e.g. teachers, 
administrators) 


Professional development aligned 

to the CCSS on the effective 
implementation of lesson plans in 
various grades and subjects, in English 
Language Learner (ELL) programs, 
special education programs including 
self-contained and co-teaching 
classrooms, and with an emphasis 

on successful practice with over-age 
under-credited students, African- 
American and Latino adolescent 
males, and girls in science and math 
classrooms 


Footage and reflections of what 
effective teaching looks like, specifically 
in alignment with the New York State 
Teaching Standards and New York 
State-approved rubrics 


Footage and reflections of what 
effective school leadership looks like, 
specifically in alignment with the 
Interstate School Leaders Licensure 
Consortium (ISLLC) standards and New 
York State-approved rubrics 


Footage of developing teachers and 
principals to be used for training 
purposes 


Highlights of highly effective teaching/ 
leadership techniques and strategies 
within the videos 


At least five instructional coaches 

to identify teachers for videotaping, 
develop CCSS-aligned lesson plans, 
apply the rubrics, and prepare students 
and teachers for filming 


See extensive detail provided in RFP. 


Online library to be used to provide 
evaluation trainees with the opportunity 
to conduct practice evaluations. Must 
include: 


+ Samples which apply a variety of 
metrics for use in calculating student 
growth and a variety of algorithms for 
weighting those metrics are necessary 
for training those districts with a 
2012-2013 implementation date; 


+ Aportion of library evaluation samples 
must be pre-scored or presented with 
sample ratings based on expert norms 
and practices for calibration purposes 
including lessons where the assigned 
ratings fall on the cusp between two 
scores; and 


+ Online resources must be regularly 
updated to allow trainees frequent 
on-demand practice opportunities to 
calibrate rater reliability. 


Technical 
Specifications 


+ Prefer to house in State's eLearning 
Delaware platform 


+ Must be secure 
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+ Calibration tools are secure, sufficiently 


deep and broad to ensure inter-rater 
reliability 


+ Tracking system to determine rater 
accuracy and reliability 


+ Integrated technology platform to 
deliver training 


+ “Professional grade video production” 


(defined on page 11 of RFP) 


+ Compatible with common video 
formats used in publicly available 
hosted sites such as YouTube 
and Vimeo 


+ Include metadata tags to all delivered 
video content aligned with the CCSS 
including, but not limited to, the 
standard adopted by the Learning 
Resource Metadata Initiative 
(www.Irmi.net) 


The web-based system must: 


Follow all ISBE Data Systems’ application 
development and coding standards 
(e.g. Microsoft®-based NET 2008 going 
against Microsoft® SQL Server 2008). All 
web-based systems run on Microsoft® 
Server 2008 R2 Data Center edition 
running IIS 6; 


All servers run as virtual machines 
under VMware ESXi 5. All backend 
storage is on an EMC VNX 5700 unified 
storage system; 


For training resources that are defined 
as “training modules,’ ISBE presently 
uses Adobe® Connect Pro and Adobe® 
Presenter to produce, publish and 
track participant progress for another 
internal ISBE division. Use ISBE existing 
structures for deliverables on the PERA 
system; 


ISBE may assign an internal 
programmer/analyst to be part of the 
development team so that the system 
can be easily transferred to in-house 
staff at the end of the contractor 
engagement; and 


For secure access to the new system, 
ISBE requires use of ISBE IWAS system 
(ISBE Web Application Security). 


The RFP term established: 


Timeframe Training and assessments delivered 
no later than the summer of 2012. All 
evaluators calibrated prior to the start 
of the 2012-2013 school year (early 


August 2012) 


Deliverables Not specifically stated 
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April 15, 2012 contract awarded 


May 15, 2012 observation rubric 
aligned to RI professional practice 
rubric 


June 1, 2012 training modules 
developed 


June 15, 2012 training modules ready 
for use with aligned videos 


July 1, 2012 training modules, videos 
and tracking systems integrated with 
the EPSS 


July 1, 2012 in each subsequent year 
will be used to update/add videos and 
establish training links to all evaluators 


Multiple training videos for evaluators 
with range of performance levels, 
grade levels and content areas 


Sample videos demonstrate a range 
of performance among competencies, 
grade level and content areas 


Training documents 
Aligned rubric 


Calibration tools that are secure, 
deep and broad to ensure inter-rater 
reliability 


Tracking system to determine rater 
accuracy and reliability 


Integrated technology platform to 
deliver training 


+ Term beginning March 30, 2012 and 
ending March 31, 2014 (vendor must 
be available immediately) 


See detailed tables provided in RFP on 
Pages 10-13. 


+ January 1, 2012 until June 30, 2013. 
ISBE, at its sole discretion, may renew 
the contract beyond the initial contract 
period. This contract may be renewed 
for up to four consecutive one-year 
contract periods or any part thereof (i.e., 
FY 2014 through FY 2017). 


+ Online library 


Delaware 
Additional Not applicable 
Requirements/ 
Qualifications 
Eligible Not stated 
Applicants 
Additional Not applicable 
background 
provided 
RFP Details Not applicable 
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+ Research-based materials: provide See detailed information provided 


evidence of effectiveness in training in RFP. 
and ability of tools to train efficiently 
and effectively and support critical 
instructional practices in the classroom 
across a broad range of grades and 
content areas. Research-based should 
include clear evidence of established 
standards for rater accuracy and 
reliability and meaningful and 
observable differences between 

score points. 


Not stated May include but not limited to: not- 


for-profit and for-profit organizations, 
companies or agencies with 
demonstrated experience in creating, 
filming and producing high-quality 
short films. Eligible applicants with 
demonstrated experience in the 
development and implementation of 
projects with similar vision and scope 
are strongly preferred. 


None provided See detailed information 


provided in RFP. 
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+ Detailed budget narrative template 


+ Describe the capacity of the bidder or 
its selected subcontractor to develop 
and maintain the web-based system. 
Address the web-based system 
technical specifications. 


All entities, including educational 
associations and institutions of higher 
education, with the necessary expertise 
in conducting educator performance 
evaluations, delivering professional 
development, providing technical 
assistance, and developing web-based 
resources, and that can demonstrate the 
qualifications required by this RFSP, are 
eligible to apply. 


+ Detailed information on State 
procurement 


« Detailed information on selection 
criteria 


* 61 pages (pages 1-19 focus on the 
content of the request) 


Sources 


This publication is based on sources of information specific to each of the four States discussed. New York is 

the only State that has a publicly available RFP: http://usny.nysed.gov/rttt/rfp/gt-1 5/nysed_rfp_gt-15.pdf. The 
information for Delaware came from the technical assistance request the State submitted to the U.S. Department of 
Education, which was not a complete RFP. Rhode Island's information also came from material shared by that State 
with the Department. Illinois had already selected a vendor at the time this publication was written, so that State's 


RFP has expired and is not publicly available. For further information about progress in developing RFPs for video 
libraries, grantees may contact info@reformsupportnetwork.org. 


This publication features information from public and private organizations and links 
to additional information created by those organizations. Inclusion of this information 
does not constitute an endorsement by the U.S. Department of Education of any 
products or services offered or views expressed, nor does the Department of 
Education control its accuracy, relevance, timeliness or completeness. 
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